Skip to content

Comments

darwin.Libsystem: fix existing file error during build#237348

Merged
toonn merged 7 commits intoNixOS:stagingfrom
reckenrode:darwin-libc-cleanup
Jun 20, 2023
Merged

darwin.Libsystem: fix existing file error during build#237348
toonn merged 7 commits intoNixOS:stagingfrom
reckenrode:darwin-libc-cleanup

Conversation

@reckenrode
Copy link
Contributor

@reckenrode reckenrode commented Jun 12, 2023

Description of changes

After coreutils was updated in #222314, darwin.Libsystem began failing to build in the rework. This appears to be due to a changing in behavior of cp -n to return a non-zero exit code when a file already exists. Note this only affects the reworked stdenv because the current one builds Libsystem with coreutils from the bootstrap tools.

Testing was done with a build of the reworked stdenv bootstrap. For aarch64-darwin, I manually built darwin.libmalloc and confirmed it was building from the newer version. Since aarch64-darwin is not building a source SDK, these headers are not used, but it is ready if/when that it does.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Jun 12, 2023
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 8.has: package (new) This PR adds a new package labels Jun 12, 2023
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jun 12, 2023
A number of headers in Libc are being vendored from other packages.
Instead of copying them from an earlier Libc, Libsystem now sources them
from their respective packages (see below). This allows Libc_old to be
dropped and avoids any potential clashes when building Libsystem.

libmalloc:
* malloc/malloc.h

libplatform:
* setjmp.h
* ucontext.h
* libkern/OSAtomic.h
* libkern/OSCacheControl.h

libpthread:
* pthread*.h
* sched.h
* spawn.h

syslog (vendored because only one file is needed):
* asl.h

xnu:
* spawn.h (a different one from libpthread)
* libproc.h
@reckenrode reckenrode force-pushed the darwin-libc-cleanup branch from 5303924 to 23cb6c1 Compare June 12, 2023 21:12
Copy link
Contributor

@toonn toonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To reproduce the error we need to rebuild darwin.Libsystem with Coreutils 9.3. These aren't an explicit dependency but come from stdenv. As reckenrode pointed out, the stdenv, probably erroneously, inherits Coreutils from the previous stage in each stage. This means stdenv just has the Coreutils from the bootstrap tarball at the point where darwin.Libsystem gets built. The final stdenv does have the proper Coreutils tools but similarly inherits darwin.Libsystem so it never ends up being built with the latest Coreutils.

However, since the final stdenv does have the correct version of Coreutils we can just rebuild against that, nix-build --pure --no-out-link -E 'with (import ./. {}); darwin.Libsystem.override { inherit stdenv; }.

@reckenrode reckenrode changed the title darwin.Libc: fix existing file error during build darwin.Libsystem: fix existing file error during build Jun 20, 2023
@reckenrode
Copy link
Contributor Author

@toonn I updated the PR to reflect that this is a Libsystem issue caused by Libc rather than a Libc issue alone. I also tested using the command you provided and can confirm it reproduces the error using the current stdenv.

$ nix-build --pure --no-out-link -E 'with (import ./. {}); darwin.Libsystem.override { inherit stdenv; }' --system x86_64-darwin
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/85zm06baw94yi7cz4nmr9yynqdrsigcw-Libsystem-1238.60.2.tar.gz
source root is Libsystem-Libsystem-1238.60.2
setting SOURCE_DATE_EPOCH to timestamp 1633495754 of file Libsystem-Libsystem-1238.60.2/xcodescripts/linker_arguments.sh
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "installPhase" }
installing
cp: not replacing '/nix/store/g65n8z3fgvy8dld580d53nqakv43g7ka-Libsystem-1238.60.2/include/./setjmp.h'

Copy link
Contributor

@toonn toonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Builds both with and without the override when on a4d88ee it only built without the override.

@toonn toonn merged commit 62f7591 into NixOS:staging Jun 20, 2023
@Artturin Artturin mentioned this pull request Aug 30, 2023
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: darwin Running or building packages on Darwin 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants